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Abstract 

The integration of nonlinear neural network methods 
with conventional linear regression techniques is 
demonstrated for representative wind tunnel force 
balance data modeling. This work was motivated by 
a desire to formulate precision intervals for response 
surfaces produced by neural networks. Applications 
are demonstrated for representative wind tunnel data 
acquired at NASA Langley Research Center and the 
Arnold Engineering Development Center in 
Tullahoma, TN. 


Introduction 

Wind tunnel testing technology activities conducted 
and sponsored by NASA Langley Research Center 
since 1997 have focussed on the introduction of 
formal experiment design principles to empirical 
aerodynamics 1 . These activities have served to bring 
to bear on aerodynamic research problems the 
powerful machinery of formal experiment design first 
introduced by R.A. Fisher and associates early in the 
20 th century 2 and used successfully since then in a 
wide range of industrial, scientific, and engineering 
applications. Collectively, these methods are 
described at NASA Langley as Modern Design of 
Experiments (MDOE), after a phrase from the 
literature of formal experiment design that 
distinguishes these methods from what is commonly 
called classical experiment design. 

Classical experimentation methods have been popular 
for hundreds of years, and form the basis of 
conventional wind tunnel testing procedures in use 
today. The defining feature of classical testing 


methods is an error control strategy that requires each 
independent variable to be changed one at a time, 
while holding all other variables constant. This 
method, formally described in the literature of 
experiment design as One Factor at a Time (OFAT) 
testing, typically involves changing the levels of the 
independent variable under study as a monotonically 
increasing function of time. This is the basis of the 
common polar , for example, which is a popular wind 
tunnel testing data structure that consists of a series 
of angle of attack levels set sequentially in a 
monotonically increasing sequence, with all other 
independent variables (Mach number, angle of 
sideslip, etc) held constant. 

MDOE practitioners recognize certain weaknesses in 
the OFAT testing philosophy that can be overcome 
by formal experiment design methods that essentially 
defend against these shortcomings 3 . Contrary to 
assumptions inherent in OFAT testing, we never 
actually “hold everything else constant” when we 
change one variable at a time in a wind tunnel test. 
We simply make no intentional changes in the other 
independent variables. Changes do occur, 
nonetheless. For example, flow angularity 
fluctuations cause effective unscheduled changes in 
the angle of attack, blockage effects can cause an 
equivalent shift in Mach number, and cross-flow 
temperature gradients can induce assorted variations 
(especially in cryogenic high Reynolds number 
testing). Instrumentation and data systems drift over 
time, subtle differences can exist in procedures and 
techniques from one operator crew to another, etc, 
etc. Despite the best efforts to keep these factors 
constant, variations in individual data points are 
inevitable. 
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A central weakness of OFAT testing is that it is 
single-point oriented, so the combined effects of all 
inevitable failures to “hold everything else constant” 
are directly reflected in the errors that exist in each 
individual data point. By contrast, MDOE methods 
are “data-ensemble” oriented. The data set is 
analyzed as an integrated whole, rather than on a 
single-point basis. The focus is on defining 
relationships between each response variable and the 
independent variables that cause it to change. If 
these relationships are well understood, it is possible 
to predict responses for any combination of 
independent variables in the range of those tested, not 
simply the specific points set in the test. 

Furthermore, such mathematical relationships (called 
response surface models) can be used to predict 
responses in other circumstances, e.g. in other tests or 
in flight. By focusing on the response surface as a 
whole rather than individual data points, and by 
adopting tactics intended to ensure that all errors are 
identically and independently distributed, the effects 
of local single-point errors can be made to 
substantially cancel. This results in higher precision 
than can be achieved on a single-point basis. 
Furthermore, the residual degrees of freedom - those 
points in excess of the minimum required to define 
the response surface - are available to assess the 
quality of predictions made by the model. 

The most common method for defining response 
surface models in MDOE wind tunnel testing to date 
has been linear regression. Linear regression results 
in low-order polynomial functions of (possibly 
transformed) independent variables that can produce 
adequate predictions for a wide range of practical 
applications. This type of model is well suited for 
limited ranges of the independent variables, but in 
complex situations that can occur in certain 
aerodynamic testing this may not cover the entire 
range of interest. For response relationships rich in 
higher-order components, such as those featuring the 
near-discontinuous responses that are not uncommon 
in transonic aerodynamics, for example, extensions to 
conventional low-order linear regression will be 
needed to produce response surfaces adequate for 
reliable prediction. One promising possibility is the 
application of neural network methods to nonlinear 
response problems. 

In the present paper, we have replaced the 
polynomial model by a Neural Network in 
constructing the Response Surface. The range of test 
design points could be enlarged to cover most test 
variables required range since the neural networks are 
capable to map a highly nonlinear hyper-surface. The 
identification of underlying model could also be 


studied from the response surface. In addition to data 
interpolation performed by the neural networks, the 
analysis of derivative functions, variable sensitivity 
and interaction effects could be investigated even 
though we have not included in the present effort. 

The widely used algorithms of neural networks in 
mapping function or constructions of response 
surfaces are the "Back Propagation" and "Radial 
Basis Function Networks". Since the model of neural 
networks is nonlinear in nature, the confidence and 
prediction interval analysis is not feasible by adapting 
the linear regression approach based on the statistical 
theorem. However, the importance of the confidence 
and prediction intervals of a constructed response 
surface is recognized for the MDOE application. 
Without the capability to estimate confidence or 
prediction intervals, the neural network is unable to 
provide the fitting goodness characteristics or the 
imperfection in the model. The confidence interval is 
the way to identify the systematical errors in the 
model or the model adequacy. Furthermore, the 
prediction confidence interval is able to estimate or 
forecast the uncertainty of response surfaces of the 
future observations. This is particular important to 
know the expected uncertainty since the data are not 
even available in a region of interest. Therefore, the 
effort of this work has been devoted to construct a 
special type of neural networks including the feature 
of linear regression to compute the response surface 
and its confidence and prediction intervals. 


Radial Basis Function Network Application to 
Tunnel Data of the Alpha-Jet Model 

Two neural network algorithms, which are capable of 
modeling response surfaces, have been selected to 
integrate with multiple linear regression to compute 
precision intervals. In the first approach, the Radial 
Basis Function Network (RBFN) is introduced to 
compute precision intervals. A brief description of 
Radial Basis Function Network and the integrated 
computation procedure of Confidence and Prediction 
intervals with the linear regression analysis are given 
in Appendix 1. As an example, the application to 
modeling force data as a function of angles of attack 
for the TST Alpha Jet Model as shown in Figure 1 is 
included for data 4 acquired from Tunnel 16T at the 
Arnold Engineering and Development Center 
(AEDC) 

The force coefficients 4 were taken at Mach Number 
0.8 and Chord Reynolds Number 1.5 millions under 
transition-free configuration in the present 
application. The angle of attack ranges from -4 to 10 
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degree. The results obtained from the RBFN and the 
original data are plotted in Figure 2. The comparison 
of RBFN results and tunnel data is within the 
accuracy of tunnel measurement. The results of 95% 
Confidence Interval Half Width on the response 
surface from Eq. (1-4) are also shown in Figure 2 as 
the error band by the Coef-HI (or -UPPER) and Coef- 
LO (-LOWER). The results are satisfactory as 
expected. In Figure 3, the 95% CIHW (Confidence 
Interval Half Width) is and Coef-Residual, which is 
defined as 

Coef-Residual = Coef-RFBN - Coef-DATA 

are plotted for the purpose of comparison. The 
prediction interval was not computed in this example 
since we do not have sufficient data to reserve as the 
test file. The prediction interval formula will be 
applied in the second example. 

Back Propagation Neural Network Application to 
Tunnel Data of the Alpha Jet Model. 

In the second approach, the integration of back 
propagation neural networks and multiple linear 
regression has been constructed to compute response 
surface and the precision confidence interval. 

Although the RBFN is capable to map multiple 
variable function with the rapid training process, the 
majority of function mapping has been carried out by 
Back Propagation (BP) Neural Networks. It is well 
known that the BP Neural Network has the powerful 
capability of function mapping to model the 
nonlinear response surface in a large numbers of 
parameters in a wide range. Thus the determination 
of the precision intervals for the BP nets is also 
necessary in the function mapping of neural networks 
application. 

The typical back propagation network has an input 
layer, an output layer and one or more hidden layers. 
The network relationship is a non-linear function. 
The analysis of confidence and prediction intervals 
based on the statistical method is not available. The 
concept of integration of the linear regression method 
into the last hidden layer of the back propagation 
network (i.e. the hidden layer just before the output 
layer) is enable to map the response surface but also 
to evaluate the confidence and prediction intervals of 
the response surface. This is a special case of linear 
regression model as named in Ref. 5. The integration 
of the process is described as follows. 


The first step is to train a selected design of a back- 
propagation neural network for the desired response 
surface with its inputs. After the network is 
satisfactorily trained, the linear regression will be 
incorporated in the trained BP nets. The processing 
elements of the last hidden in the nets will be treated 
as regressor variables in a multiple linear regression 
model. With the trained weights of the BP nets, each 
observation will provide the value of regressor 
variables, 4>„ and the response, F, known as the 
desired output for each input data set as 

K 

F = 'E l w ,<t>, + w o+ e 

/=l 

Where the <)), 's are regressor variables functions of 
the input layer data and all weights in the trained 
Back Propagation (BP) Neural Network. The w, ’s are 
known as the regression coefficients. The error (or 
residual) of the regression is e. The input variables 
in the cases of wind tunnel data include Mach 
number, Reynolds number, Angle of Attack and so 
on. The outputs, F, are force coefficients, e.g. lift, 
drag and pitching moment, for model force data. 

In addition, a modified Back Propagation Net can be 
enhanced by introducing functions in terms of input 
variables linearly independent. The modified net, 
which is called as Functional-link nets 6 , is to enhance 
the original representation of input. The additional 
dimensions produced by these functions may be 
learned more readily in the hyperplanes. These 
functions typically consist of outer-product and 
functional enhanced modes of input variables. Some 
of the superior qualities of Functional-link nets have 
been demonstrated in the supervised train net by 
many examples in the literature. This technique of a 
simple representation of the net is illustrated in 
Figure 4. The Functional-link Back Propagation nets 
have been applied in the present work to map all 
force coefficients of a force model in the next 
section. 

This integrated method applied to tunnel force data of 
the Alpha Jet Model with three typical variables, 
Mach number, Reynolds Number and Angle of 
attack, which consists of 25 test configurations 
conducted in the National Transonic Facility at 
NASA/Langley Research Center. Each test 
configurations ranges from 10 to 22 points of angles 
of attack. The total number of test points is over four 
hundred sets of data. The integrated method is able to 
map all force data into a single neural network. The 
confidence and prediction intervals associated with 
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this neural network are computed by Eq.(I-4) and 
Eq.(I-6) for the Alpha Jet Model data, respectively. 

In the present investigation, the data ranges of Mach 
Number from 0.6 to 0.9 and Chord Reynolds number 
from 2.7 to 10 millions for transition-free 
configurations are available. The angle of attack 
ranges from -4 to 10 degree for most test conditions. 

With half of all database for the training cases of the 
neural network, the remaining half set of data are 
reserved for the testing cases. The force and moment 
coefficients plots of a typical training case for Mach 
Numbers 0.8 are shown in Figure 5. It can be seen 
that the comparison of Neural Network-LSM 
prediction and tunnel data is good for those typical 
training flow conditions. The Coef-HI (-Upper) and 
Coef-LO (-Lower) are also plotted in Figure 5 for the 
range of the 95% confidence intervals. The 95% 
CIHF (confidence interval half-width) and Coef- 
Residual at tunnel condition M=0.8 are plotted in 
Figure 6. The results are satisfactory. 


Concluding Remarks 

Multiple linear regression methods have been 
integrated into two neural network algorithms, the 
Radial Basis Function Network and the Back 
Propagation Network. Both neural networks, which 
have nonlinear characteristics, are capable of 
constructing nonlinear response surfaces from the 
wide range of data sets obtained in typical MDOE 
applications. Response surface precision intervals 
(confidence and prediction intervals) are determined 
by linear regression analysis. Applications of the 
Radial Basis Function Network and Back 
Propagation integrated method to the force data sets 
of an Alpha Jet Model obtained from AEDC Tunnel 
16T and NASA Langley NTF have shown 
satisfactory data mapping results. These results 
suggest that there is significant potential for neural 
networks to be applied in wind tunnel testing. 

With their non-linear adaptive training and cross- 
validation capabilities integrated with formal, 
quantitative goodness-of-fit assessment techniques, it 
may be possible to largely automate the examination 
of complex, non-linear aerodynamic phenomena. 
MDOE methods can be used to select information- 
rich combinations of independent variables that are 
then used to train a neural network. Predictions by 
the network can be compared to an independent 
confirmation data set to determine if an adequate 
percentage of confirmation points lie within the 
precision interval of the response surface. If not, 


additional training data can be acquired until 
adequate predictions are demonstrated throughout the 
inference space of interest. This has the potential of 
simultaneously defining prediction models that 
satisfy design precision requirements while ensuring 
that no more data are acquired than is necessary to do 
so. 
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Appendix I 

The Radial Basis Function Networks 


The Radial Basis Function Network (RBFN) with N 
inputs and a scalar output, which is depicted in 
Figure 1-1, can be expressed for a function 
approximation as 


f{ x ,i w )=Yj W ' ~ c «f 

7=1 


l+W 


(H) 


V 


Where x } 's are the inputs, <(>, ’s are the given basis 
function and w, f s are the weights. The Gaussian 
function is chosen as the Basis Function as shown in 
Figure 1-1. The Gaussian Function with a radial-basis 
function argument that is used to form a network is 
called Radial Basis Function Networks (RBFN). The 
Gaussian function of the input variable x } 's is of the 
form 
Where 



c.. 's are the centers or mean values and ~ 

7=1 

standard deviation of a normal distribution function 
of statistics. 


By specifying a set of inputs, x, ’s and the 
corresponding desired output F, the values of the 
weights Wj 's can be determined using the linear Least 
Squares Method (LSM). 

The above-described RBFN is a special case of 
Multiple Linear Regression models. The F is the 
desired output and is called as the Response. The <j>, is 
known as regressors, which are a specified function 
of inputs Xj 's. 

The pattern unit (or regressors) in a RBF Network 
consists of center, c y and deviation, a, for each 
Gaussian function. A clustering algorithm is applied 
by Moody & Darken 7 to determine the value of 
centers and a nearest neighbor heuristic to determine 
the deviation, a,. The Linear Regression or a 
gradient descent algorithm evaluates the weights of 


the output function. The linear regression will be 
used in the present application. Therefore the 
confidence interval and predictive confidence interval 
can be determined by the available statistical method 
for this radial basis function network. 

The description given herein has only a single 
(scalar) output for notational simplicity. There is no 
limitation of number of outputs. To extend the 
multiple outputs, another sets of weights should be 
introduced for additional desired outputs. 


Implementation of RBFN and Computation of 
Precision Intervals 

The RBFN algorithm is based on Moody and Darken 
work 7 . As the trained RBNF is accepted, the values 
of the regressors are determined for specified inputs 
and the target output. With these sets of inputs and 
corresponding outputs, the Precision Intervals for 
mean response can be obtained by the linear 
regression analysis. The Confidence Interval and 
Prediction Interval are able to compute that are based 
on the standard formulae given in the linear 
regression references (e.g. Ref. 8). The formulae are 
listed as follows: 

A 100(1 -a) percent half-width of confidence 
interval (CIHW) on the mean response at a 
particular point 0//, 0/j, 0u 0/* is expressed as 


w,* 

Where t a >„-/» is t-distribution quantiles, a 2 is sum 
of squares residual/degree of freedom and 

*1 = [K 01 P012 •> 01 /r 1 

(1-5) 

X' = [x^x 2 , xj 

A 100(l-a) percent half-width of prediction interval 
(PIHW) for the future observation is of the form 


tan.n-r 0 0 + JC,'(JrX)-'x, 


( 1 - 6 ) 
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Figure 1-1 . Structure of a Radial Basis Function Network 
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Appendix 11 



Figure 1 . Transonic Technology Wing (TST Alpha Jet) Model 
From Ref 4.: Laster, ML., Stanevvsky, E, Sinclair, D.W. and Sickles, W.L., 
"Reynolds Number Scaling at Transonic Speeds," AIAA paper 98-2878, 1 998 
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Figure 2(a) Lift Coefficient CL 
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M=0.8 Re=1.5 Mil 
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Figure 2(b). Drag Coefficient, CD 
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Figure 2(c). Pitching Moment Coefficient, CLM 

Figure 2. Confidence Intervals with RBFN and DATA at M=0.8 and Re=1.5 Million. 
Note: The Coef-HI and Coef-LO are indicated the ranee of the 95% Confidence intervals. 
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CL-Residual 




Pitching Moment Coefficient - 95% Confidence Interval Half-Width 
and CLM-Residual, 16T Data 
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Figure 3(c). Pitching Moment Coefficient, RMS (CLM-Residual)=0.0005 

Figure 3. The 95% Confidence Interval Flalf-Width and RBFN Coef-Residuals. 
Data Uncertainty for Tunnel 16T ACL=0.0048, ACD=0.0009 & ACLM=0.0025. 
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Figure 4. A Functional-Link Back Propagation Neural Network with Regular Tunnel Variable Data Inputs 
and Underlying Model Functions in terms of Regular Data as Additional Inputs. 
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Figure 5 (c). Pitching Moment Coefficient 

Figure 5. Training Results: Comparison of Neural Networks-LSM Prediction and Tunnel Data at Mach 0.8 Reynolds 

Number 3.3 mil for Force Coefficients. 

Note: Coef-LO and Coef-Hl are 95% Confidence Intervals for Neural Networks-LSM *s Response Surface. 
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